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REMARKS 

Claims 13, 16-18, 20-23, 25, 26, 29-35 and 37-43 are currently pending in 
this application. Claims 13 and 16 have been amended and claim 19 has been 
cancelled. No new matter has been added to this application. 

Rejection of Claims 13, 16-18. 20-22. 42 and 43 under 35 U.S.C. 6 112 

The Examiner has rejected claims 13, 16-18, 20-22, 42 and 43 under 35 
U.S.C. § 112, second paragraph, as being indefinite. Applicants respectfully 
traverse the rejection. 

Applicants have amended claims 13 and 16 to remove any references to 
"it" and have canceled claim 19. Claims 17, 18, 20-22, 42 and 43 being 
dependent upon independent claim 16 are believed to be definite. Applicants 
respectfully submit that claims 13 and 16, as amended, correct any prior 
indefiniteness and request that the rejection of the claims under 35 U.S.C. § 1 12, 
second paragraph be withdrawn. 

Rejection of Claims 13. 16-18. 22. 23. 31-35. 37 and 40-42 under 35 U.S.C. § 
103(a) 

The Examiner has rejected claims 13, 16-18, 22, 23, 31-35, 37 and 40-42 
under 35 U.S.C. § 103(a) as being unpatentable over U.S. Patent Publication No. 
2002/077540 (Kienzle) in view of Frank Sauer et al., "Augmented Workspace: 
designing an AR testbed" (Sauer). The Examiner contends that Kienzle 
discloses a method for augmented reality guided instrument positioning. The 
Examiner correctly notes that Kienzle does not teach or disclose rendering at 
least one graphics path guide on a display overlaid onto a direct optical or video 
view that contains the real instrument and a real object, which includes the entry 
point. The Examiner contends that Sauer teaches a video see through HUD and 
superimposing graphics guides onto the video of the real instrument and the real 
object. The Examiner argues that it would have been obvious to one of ordinary 
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skill in the art to display the rendered graphics path guide on Kienzle's display 
overlaid onto a direct optical or video view that contains the real instrument and 
the real object. Applicants respectfully traverse the rejection. 

Applicants' application currently recites three claim sets, each directed to 
a different embodiment of the present invention. Claim 13 is directed to a 
method for augmented reality guided instrument positioning in which at least one 
graphics path guide is rendered for indicating a path for a real instrument to 
follow to a target located inside or on the surface of a real object. The graphics 
path guide is constructed in a way that the frame guide frames the path at least 
outside of the real object so as not to obstruct a view of a central part of the real 
instrument along the real instrument's axis as viewed from the side if the real 
instrument is in correct alignment to the path. The rendered at least one 
graphics path guide is displayed on a display overlaid onto a direct optical or 
video view that comprises a real view of the real instrument and the real object. 
A user moves the real instrument, observed in the real video view, to align the 
real instrument with the at least one graphics path guide. The display shows the 
at least one graphics guide overlaid onto the view of the real instrument and the 
real object. The real instrument is aligned with the path by determining when the 
at least one graphics path guide frames the path so that a view of a central part 
of the real instrument is not obstructed by the at least one graphics path guide. 
The real instrument is moved along the path so that a front portion of the real 
instrument is inserted into the object until the real instrument's tip reaches the 
target, and concurrently monitoring the correct alignment between the visible part 
of the real instrument outside of the real object and the framing path guide. 

Claims 16-18, 20-22, 42 and 43 are directed to a method for 
augmented reality guided instrument positioning in which a bulls eye graphics 
path guide is used to align an instrument with an entry point on a real object 
corresponding to a target. A target point is defined within an real object. A path 
is defined to reach the target point with a real instrument. A graphical 
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representation is rendered to mark the path in the form of at least one graphical 
axis marker to be viewed in the direction of the path towards the target on a 
display overlaid onto a direct optical or video view of a real scene that comprises 
the real instrument, and the real object, so that a user can perceive the real 
instrument and the at least one graphical axis marker and their spatial 
relationship in an augmented reality view. A user's augmented reality line-of- 
sight is aligned with the at least one graphical axis marker so that the path to 
reach the target point with the real instrument is aligned along the augmented 
reality line-of-sight. The real instrument is aligned to the path by aligning the real 
instrument with the augmented reality line-of-sight towards the target point. The 
real instrument is moved along the path towards the actual target point while 
keeping the real instrument aligned with the augmented reality line-of-sight. 

Claims 23, 25, 26, 29-35 and 37-41 are directed to a method for virtual 
reality guided instrument positioning. A target is defined on a real object. A path 
to reach the real object is defined. A pose of a real instrument with respect to a 
pose of the real object is tracked. A graphical representation of the real 
instrument and the path is rendered to obtain a virtual instrument and a graphical 
virtual guide on a display. . The graphical representation is rendered with 
respect to a virtual viewpoint from which a virtual line of sight coincides with a 
virtual path for the virtual instrument to follow during a positioning of the real 
instrument to the target point. The virtual guide corresponds to the path. The 
virtual instrument comprises a 3D structure for line of sight alignment. The 3D 
structure comprises a plurality of markers centered on and distributed along an 
axis of the virtual instrument; 

The virtual instrument is aligned along the virtual line of sight according to 
the virtual guide in order to accordingly align the real instrument along the path. 
The real instrument is moved by a user in response to viewing the virtual 
instrument and the virtual guide on the display. The real instrument moves along 
the path towards the real target point keeping the correct alignment by observing 
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and keeping in alignment with the virtual instrument and virtual guide. The user 
needs to keep the alignment intact while inserting the real instrument into the real 
object toward the target point. Claim 23 introduces tracking of the real 
instrument. In this case, position and orientation of the real instrument with 
respect to the real object and the path are known, and the user does not need an 
augmented reality view that comprises a view of the real instrument to perform 
correct alignment. Alignment can now be performed based on a graphical virtual 
representation of both instrument and path. 

Kienzle discloses a computer assisted surgery system which uses graphic 
representations that are displayed overlaid onto X-ray images. Kienzle discloses 
a standard navigation system in which the location of the tracked instrument is 
superimposed as a virtual model onto a medical image. Correct alignment is 
achieved by looking at the position of the instrument's virtual model in the 
medical image. In Kienzle, using a tracking system to track the position of the 
actual instrument is an essential part of the navigation process. 

In the present invention, there is no need to track the actual instrument 
(except claim 23, which uses tracking in combination with the specific line-of- 
sight method). A user sees a real view of the real instrument and a real patient. 
These real images are augmented with graphical information that allows the user 
to perceive planned paths for instrument guidance. Correct alignment is 
performed by looking at the augmented reality image and seeing the position of 
the real instrument in relation to the graphical guides. Applicants respectfully 
submit that Kienzle does not teach using augmented reality for guided instrument 
positioning. Kienzle merely teaches using a virtual feature to identify landmark 
points in the body of a patient. Applicants' method uses a graphical guide to 
align with an entry point, which in turn is lined up with a target. However, unlike 
Kienzle, Applicants do not use markers or any type of imaging to see points 
within the body. Applicants' method is performed using a real world perspective; 
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the user does not see images within the body. As such, Kienzle does not teach 
or disclose Applicants' invention as claimed. 

The Sauer article shows the work that was done prior to Applicants' 
invention. Sauer shows using augmented reality to guide a needle in which a 
"virtual cylinder indicates the correct needle path in 3D". However, Sauer does 
not teach or disclose using a frame to guide the path as disclosed in claim 13 in 
which part of the needle is exposed. Sauer uses a cylinder which completely 
obstructs the vision of the needle during alignment. It has been found that this 
can cause improper placement of the needle because it is not clear when the 
needle is obstructed if alignment has been reached or if the needle is in the 
incorrect orientation. The frame serves the dual purpose of guiding the needle 
while allowing the user to confirm the proper alignment of the needle. 

Likewise, Sauer does not teach or disclose using a bull's eye approach to 
guide the needle to an entry point to a target. In Figure 7, the user is viewing the 
cylinder lengthwise and trying to align the needle with the cylinder. Sauer is not 
using the cylinder in a bull's eye fashion as is recited in independent claims 16 
and 23. Applicants respectfully submit, that Sauer does not teach or disclose 
Applicants' invention as claimed. 

Applicants respectfully submit that the combination of Kienzle and Sauer 
do not teach or disclose Applicants' invention as claimed. Neither Kienzle nor 
Sauer, whether taken alone or in combination, teach or disclose a graphics guide 
which frames the path of an instrument to an entry point corresponding to a 
target. Nor do Kienzle or Sauer teach or disclose a graphics line-of-sight guide 
which guides an instrument to an entry point corresponding to a target. 
Applicants request that the rejection of claims 13, 16-18, 22, 23, 31-35, 37 and 
40-42 under 35 U.S.C. § 103(a) be withdrawn. 
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Rejection of Claim 20, 21, 25. 26. 29. 30. 38. 39 and 43 under 35 U.S.C. § 
103(a) 

The Examiner has rejected claims 20, 21, 25, 26, 29, 30, 38, 39 and 43 
under 35 U.S.C. § 103(a) as being unpatentable over Kienzle in view of the 
Sauer article and in view of Applicants' admitted prior art. The Examiner 
correctly notes that Kienzle does not teach or disclose a graphical axis marker 
that comprises an intersection of at least two lines that is centered on the axis of 
the actual instrument for correct alignment. Nor does Kienzle teach rendering by 
using a virtual camera with a wide angle lens. The Examiner contends that Sauer 
teaches uses gun aiming and argues that it would have been obvious to one 
skilled in the art to apply it to augmented reality aiming of real instruments and 
using a virtual camera with a wide angle lens. Applicants respectfully traverse 
the rejection. 

Applicants respectfully submit that the combination of Kienzle, 
Sauer and the prior art cited by Applicants does not teach or disclose Applicants' 
invention as claimed. Neither Kienzle nor Sauer nor the prior art teach or 
disclose an augmented reality method for positioning an instrument in which the 
steps of "aligning the user's augmented reality line-of-sight with the at least one 
graphical axis marker so that the path to reach said entry point with the real 
instrument is aligned along the augmented reality line-of-sight" and "aligning the 
real instrument to the path by aligning the real instrument with the augmented 
reality line-of-sight towards the entry point;" as recited in amended independent 
claim 16. Claims 20 and 21 being dependent upon independent claim 16 are 
also not taught or disclosed by the combination of Kienzle, Sauer and the prior 
art. Likewise, neither Kienzle, Sauer or the prior art teach or disclose "aligning 
the virtual instrument along the virtual line of sight according to the graphical 
virtual guide in order to accordingly align the real instrument along the path" and 
"moving the real instrument by a user in response to viewing the virtual 
instrument and said graphical virtual guide on the display, the real instrument 
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moving along the path towards the real entry point keeping the correct alignment 
by observing and keeping in alignment with the virtual instrument and graphical 
virtual guide" as recited in independent claim 23. Claims 25, 26, 29, 30, 38, 39 
and 43 being dependent upon independent claim 23 are also not taught or 
disclosed by the combination of Kienzle, Sauer and the prior art. Applicants 
respectfully request that the rejection of claim 20, 21 , 25, 26, 29, 30, 38, 39 and 
43 under 35 U.S.C. § 103(a) be withdrawn. 

Conclusion 

Applicants respectfully submit that claims 13, 16-18, 20-23, 25, 26, 29-35 
and 37-43, as amended, are in condition for allowance and request that a timely 
Notice of Allowance be issued in this case. The Examiner is invited to contact 
the undersigned should he have any questions in this matter. 



Date: January 15.2008 
Siemens Corporation 
Intellectual Property Department 
1 70 Wood Avenue South 
Iselin, New Jersey 08830 
(732) 321-3013 



Respectfully submitted, 



Michele L. Conover 
Reg. No. 34,962 
Attorney for Applicant 
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